Algebraic Analysis for Non-regular Learning Machines

Hierarchical learning machines are non-regular and non-identifiable statistical models, whose true parameter sets are analytic sets with singularities. Using algebraic analysis, we rigorously prove that the stochastic complexity of a non-identifiable learning machine is asymptotically equal to λ1 log n − (m1 − 1) log logn + const., where n is the number of training samples. Moreover we show that the rational number λ1 and the integer m1 can be algorithmically calculated using resolution of singularities in algebraic geometry. Also we obtain inequalities 0 < λ1 ≤ d/2 and 1 ≤ m1 ≤ d, where d is the number of parameters.